Generalized Higher-Order Tensor Decomposition via Parallel ADMM

Higher-order tensors are becoming prevalent in many scientific areas such as computer vision, social network analysis, data mining and neuroscience. Traditional tensor decomposition approaches face three major challenges: model selecting, gross corruptions and computational efficiency. To address these problems, we first propose a parallel trace norm regularized tensor decomposition method, and formulate it as a convex optimization problem. This method does not require the rank of each mode to be specified beforehand, and can automatically determine the number of factors in each mode through our optimization scheme. By considering the low-rank structure of the observed tensor, we analyze the equivalent relationship of the trace norm between a low-rank tensor and its core tensor. Then, we cast a non-convex tensor decomposition model into a weighted combination of multiple much smaller-scale matrix trace norm minimization. Finally, we develop two parallel alternating direction methods of multipliers (ADMM) to solve our problems. Experimental results verify that our regularized formulation is effective, and our methods are robust to noise or outliers. Introduction The term tensor used in the context of this paper refers to a multi-dimensional array, also known as a multi-way or multi-mode array. For example, if X ∈ RI1×I2×I3 , then we say X is a third-order tensor, where order is the number of ways or modes of the tensor. Thus, vectors and matrices are first-order and second-order tensors, respectively. Higher-order tensors arise in a wide variety of application areas, such as machine learning (Tomioka and Suzuki, 2013; Signoretto et al., 2014), computer vision (Liu et al., 2009), data mining (Yilmaz et al., 2011; Morup, 2011; Narita et al., 2012; Liu et al., 2014), numerical linear algebra (Lathauwer et al., 2000a; 2000b), and so on. Especially, with the rapid development of modern computer technology in recent years, tensors are becoming increasingly ubiquitous such as multi-channel images and videos, and have become increasingly popular (Kolda and Bader, 2009). When working with high-order tensor data, various new computational ∗Corresponding Author Copyright c © 2014, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. challenges arise due to the exponential increase in time and memory space complexity when the number of orders increases. This is called the curse of dimensionality. In practice, the underlying tensor is often low-rank, even though the actual data may not be due to noise or arbitrary errors. Essentially, the major component contained in the given tensor is often governed by a relatively small number of latent factors. One standard tool to alleviate the curse is tensor decomposition. Decomposition of high-order tensors into a small number of factors has been one of the main tasks in multi-way data analysis, and commonly takes two forms: Tucker decomposition (Tucker, 1966) and CANDECOMP/PARAFAC (CP) (Harshman, 1970) decomposition. There are extensive studies in the literature for finding the Tucker decomposition and the CP decomposition for higherorder tensors (Kolda and Bader, 2009). In those tensor decomposition methods, their goal is to (approximately) reconstruct the input tensor as a sum of simpler components with the hope that these simpler components would reveal the latent structure of the data. However, existing tensor decomposition methods face three major challenges: rank selection, outliers and gross corruptions, and computational efficiency. Since the Tucker and CP decomposition methods are based on least-squares approximation, they are also very sensitive to outliers and gross corruptions (Goldfarb and Qin, 2014). In addition, the performance of those methods is usually sensitive to the given ranks of the involved tensor (Liu et al., 2009). To address the problems, we propose two robust and parallel higher-order tensor decomposition methods with trace norm regularization. Recently, much attention has been drawn to the low-rank tensor recovery problem, which arises in a number of realword applications, such as 3D image recovery, video inpainting, hyperspectral data recovery, and face reconstruction. Compared with matrix-based analysis methods, tensorbased multi-linear data analysis has shown that tensor models are capable of taking full advantage of the high-order structure to provide better understanding and more precision. The key idea of low-rank tensor completion and recovery methods is to employ matrix trace norm minimization (also known as the nuclear norm, which is the convex surrogate of the rank of the involved matrix). In addition, there are some theoretical developments that guarantee reconstruction Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence